Automatic personal assistance between users devices

ABSTRACT

Methods and systems are presented for providing automatic personal assistance on a user device (e.g., mobile phone). In some embodiments, a personal assistant on a first user device may respond automatically to a received input communication from a second user device. In some embodiments, the personal assistant may perform a local search of data stored on the first user device in order to retrieve data responsive to the received input. In some embodiments, the personal assistant may perform a local load of an entry into an application on the first user device in order to perform an action responsive to the received user input. In some embodiments, the automatic response may include sending a responsive output communication to the second user device. The automatic personal assistance may be customized based on user-editable automatic personal assistance settings.

CROSS-REFERENCE TO RELATED APPLICATIONS

The present invention claims priority benefit to U.S. provisional patentapplication No. 62/007,933, filed Jun. 5, 2014, the entirety of which isincorporated by reference.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention generally concerns a personal assistant feature ona user device. More particularly, the present invention concernsproviding automatic personal assistance on a user device.

2. Description of the Related Art

A digital intelligent personal assistant software application (e.g.,personal assistant) is commonly run on a smart phone or intelligentdevice. Personal assistant applications may respond to user input,including, for example, voice-based input, often taking the form of aquery or command. Based on the received user input, a personal assistantapplication may perform tasks by utilizing location awareness andaccessing information stored on the smart device and web services. Indoing so, the application appears to answer questions, makerecommendations, and perform other actions. A personal assistant mayperform pointed actions in response to user commands, including, forexample, sending a message to a particular person or scheduling aspecific meeting at a specific time in a calendar.

Existing personal assistant applications are capable of performing anincreasing number of commands, but existing personal assistantapplications do not provide for automatic interaction of two personalassistants on two user devices, where the interactions may not requireuser involvement. Thus, there exists a need to provide personalassistant features on a first user device that may be automaticallyinvoked by input received from a second personal assistant on a seconduser device.

SUMMARY OF THE CLAIMED INVENTION

Methods and systems are presented for providing automatic personalassistance on a user device (e.g., mobile phone). In some embodiments, apersonal assistant on a first user device may respond automatically to areceived input communication from a second user device. In someembodiments, the personal assistant may perform a local search of datastored on the first user device in order to retrieve data responsive tothe received input. In some embodiments, the personal assistant mayperform a local load of an entry into an application on the first userdevice in order to perform an action responsive to the received userinput. In some embodiments, the automatic response may include sending aresponsive output communication to the second user device. The automaticpersonal assistance may be customized based on user-editable automaticpersonal assistance settings.

Various embodiments may include methods for providing automatic personalassistance on a first user device. Such methods may include receiving aninput communication, at a communications interface on the first userdevice, the input communication transmitted over a network from a seconddevice. The methods may further include activating an automatic mode ofa first personal assistant on the first user device. The methods mayfurther include parsing the received input communication, by the firstpersonal assistant, to identify one or more keywords in the receivedinput communication, the one or more keywords associated with aresponsive action, wherein the first personal assistant processes thereceived input communication automatically. The methods may furtherinclude performing the identified responsive action, by the firstpersonal assistant, and generating an output communication responsive tothe input communication, by the first personal assistant, wherein theoutput communication is transmitted over the network to the seconddevice, and wherein the output communication includes informationassociated with the performed responsive action.

Various embodiments may further include systems for providing automaticpersonal assistance on a first user device. Such systems may include acommunications interface on the first user device that receives an inputcommunication transmitted over a network from a second device. Thesystems may further include a memory that stores instructions, and aprocessor that executes the instructions stored in the memory. Theprocessor may execute the instructions to activate an automatic mode ofa first personal assistant on the first user device. The processor mayfurther execute the instructions to parse the received inputcommunication, by the first personal assistant, to identify one or morekeywords in the received input communication, the one or more keywordsassociated with a responsive action, wherein the first personalassistant processes the received input communication automatically. Theprocessor may further execute the instructions to perform the identifiedresponsive action, by the first personal assistant, and to generate anoutput communication responsive to the input communication, by the firstpersonal assistant, wherein the output communication is transmitted overthe network to the second device, and wherein the output communicationincludes information associated with the performed responsive action.

Embodiments of the present invention may further include non-transitorycomputer-readable storage media, having embodied thereon a programexecutable by a processor to perform methods for providing automaticpersonal assistance on a user device as described herein.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 illustrates an exemplary network environment in which a systemfor providing automatic personal assistant features on a user device maybe implemented.

FIG. 2 is a diagram illustrating exemplary settings of an operatingsystem on a user device that may be used with a system for providingautomatic personal assistant features.

FIG. 3 is an illustrative table of automatic answers provided by apersonal assistant in response to received inputs.

FIG. 4 is an illustrative table of received inputs, local searches,local loads, and automatic outputs.

FIG. 5 is a flowchart illustrating an exemplary method for providingautomatic personal assistance on a first user device in response to aninput communication received from a second user device.

FIG. 6 is a flowchart illustrating an exemplary method for providingautomatic personal assistance on a user device.

FIG. 7 illustrates an exemplary device architecture of a user devicethat may be used with a system for providing automatic personalassistant features on a user device.

DETAILED DESCRIPTION

Methods and systems are presented for providing automatic personalassistance with a personal assistant feature of a user device (e.g.,mobile phone). In some embodiments, a first personal assistant on afirst user device may receive a personal assistance command from asecond personal assistant on a second user device. The first personalassistant may perform an action responsive to the received command basedon user settings. In some embodiments, the first personal assistant mayautomatically perform the responsive action (i.e., no user input isrequired for performance of the action). In some embodiments, the firstpersonal assistant may receive the personal assistant command, identifya responsive action, and prompt a user of the first user device forpermission before performing the identified responsive action. at aninterface of a user device, a personal assistant feature will return acustomizable status report. Personal assistant software may be anapplication loaded onto the user device by the user, or any combinationof software, firmware, and hardware that is incorporated into the userdevice. A personal assistant software application on a user device mayprovide a status report based on local status data stored locally on theuser device and/or remote status data accessed through a network.

FIG. 1 illustrates an exemplary network environment 100 in which asystem for providing automatic personal assistant features on a userdevice may be implemented. Network environment 100 may include userdevices 105 and 110, network 175, network connections 160, 165, and 170,and third party server 180. Any combination of the componentsillustrated in network environment 100, including user devices 105 and110, network 175, network connections 160, 165, and 170, third partyserver 180, and modules, processes, or subsystems of each, and any otherhardware, software, or both, for implementing the features described inthe present disclosure may be collectively referred to, herein, as “thesystem.”

User devices 105 and 110 may be any number of different electronic userdevices, including, for example, general purpose computers, mobilephones, smartphones, personal digital assistants (PDAs), portablecomputing devices (e.g., laptop, netbook, tablet), desktop computingdevices, handheld computing device, or any other type of computingdevice capable of communicating over network 175. User devices 105 and110 may also be configured to access data from other storage media, suchas memory cards or disk drives as may be appropriate in the case ofdownloaded services. User devices 105 and 110 may include standardhardware computing components, including, for example, network and mediainterfaces, non-transitory computer-readable storage (memory), andprocessors for executing instructions that may be stored in memory.

In the illustrated embodiment, user devices 105 and 110 (e.g., mobilephone) include a display. In some implementations, the display may be atouchscreen display. In some implementations, the display is a userinterface. As shown in the illustrated embodiment, a display of userdevices 105 and 110 may display icons corresponding to applications(i.e., applications 140). The displays of user devices 105 and 110 mayinclude any suitable soft keys. It will be understood that user devices105 and 110 may include other elements not shown, for example, amicrophone, camera, speaker, or any other suitable hardware or softwareelements.

User devices 105 and 110 may include respective operating systems 115and 120. Operating systems 115 and 120 may be software that manages theuse of hardware, computer programs, and applications of user device 105.Operating systems 115 and 120 may be, for example, Windows, iOS, OS X,Android, UNIX, or Linux. User device 105 may additionally includesettings 125, which may include configurable components of operatingsystem 115. User device 110 may additionally include settings 130, whichmay include configurable components of operating system 120. Settings125 and 130 may be modifiable by a user of the respective user devices105 and 110 to alter the performance of operating systems 115 and 120and other software on user devices 105 and 110. In some embodiments,settings 125 and 130 may be applications on respective user devices 105and 110, by which a user may select options and preferences andconfigures operating system functions. In an example, operating systems115 and 120 of user devices 105 and 110 (e.g., an Apple device) may beiOS, and settings 125 and 130 of user devices 105 and 110 may be iOSsettings. In another example, operating systems 115 and 120 may beLINUX, and settings 125 and 130 may be LINUX configuration files. Insome embodiments, settings 125 and 130 may include personal assistantsettings, which are modifiable by a user to alter the performance ofpersonal assistant software on user devices 105 and 110. In someembodiments, settings 125 and 130 may be modifiable by a user toconfigure the ability of personal assistant software running on one ofuser devices 105 and 110 to automatically interact with personalassistant software running the other of user devices 105 and 110.

User devices 105 and 110 may include any suitable software orapplications. In some embodiments, personal assistant software (notshown) runs on user devices 105 and 110. The personal assistant may besoftware capable of performing tasks for a user based on, for example,user input, location awareness (e.g., using a global positioningsystem), user settings 125 and 130, locally stored information (e.g., inmemory of user devices 105 and 110) and information accessible over anetwork (e.g., network 175) from a personal assistant server 180 andthird party databases (not shown). Existing, exemplary, personalassistants include, for example, SIRI® services (for Apple devices),GOOGLE NOW® services (for Google Android devices), S VOICE® (for Samsungdevices), and VOICE MATE® services, (for LG Electronics devices). Itwill be understood that the examples of existing intelligent personalassistants described herein are merely exemplary, and the system of thepresent disclosure may be implemented using any suitable hardware and/orsoftware.

In some embodiments, personal assistants are personal assistantapplications running on respective user devices 105 and 110. Personalassistant software may, for example, send messages, make telephonecalls, set reminders, make calendar appointments, retrieve data locallyor remotely, perform internet searches, or perform any other suitableactions in response to user input. In some embodiments, depressing anelectromechanical button may activate the personal assistant. In someembodiments, actuating a personal assistant soft key may turn thepersonal assistant ON or OFF. Personal assistant software of user device105 may receive a command from personal assistant software of userdevice 110. In some embodiments, personal assistant software of userdevice 105 may process the received command from personal assistantsoftware of user device 110 and automatically perform an actionresponsive to the command. For example, personal assistant software ofuser device 110 may send a command that includes a request for anappointment at a particular date and time to personal assistant softwareof user device 105, and personal assistant software of user device 105may, in response, load a new appointment to a calendar on user device105 and/or send a text message to personal assistant software of userdevice 110 confirming the appointment. In some embodiments, the abilityof personal assistants on user devices 105 and 110 to automaticallyinteract with each other is determined by user settings (i.e., settings125 and 130).

Applications 140 and 145 are software modules on respective user devices105 and 110, which may be downloaded from remote servers. Applications140 and 145 may provide additional functions for user devices 105 and110. For example, applications 140 and 145 may be any suitableapplications downloaded from, for example, Apple Inc.'s APP STORE® (forApple devices), GOOGLE PLAY® (for Google Android devices), or any othersuitable database or server. In some embodiments, applications 140 and145 may be software, firmware, or hardware that is integrated into userdevices 105 and 110.

Antennas 150 and 155 are components of user devices 105 and 110. In someembodiments, user devices 105 and 110 may use respective antennas 150and 155 to send and receive information wirelessly. For example,antennas 150 and 155 may be cellular data antennas, Wi-Fi antennas, orBLUETOOTH® antennas.

Network connections 160, 165, and 170 may include any suitable wired orwireless transmission mediums or channels through which data may becommunicated between user devices 105 and 110, network 175, and personalassistant server 180. Network connections 160, 165, and 170 may include,for example, a computer networking cable, an Ethernet cable, a cellularcommunications network, an Internet data trunk (e.g., singletransmission channel), a wireless local area network, a wide areanetwork, or a telecommunications network (e.g., 4G wireless network).

Network 175 may include the Internet, a system of interconnectedcomputer networks that use a standard protocol, a dispersed network ofcomputers and servers, a local network, a public or private intranet,any other coupled computing systems, or any combination thereof. In someembodiments, network 175 may be a cloud, which is a network of remoteservers hosted on the Internet and used to store, manage, and processdata in place of local servers or personal computers. User devices 105and 110 may be coupled to network 175 though any suitable wired orwireless connection. In some embodiments, user device 105 may be coupledto network 175 via network connection 160, and user device 110 may becoupled to network 175 via network connection 165.

Network 175 may allow for communication between the user devices 105 and110, and personal assistant server 180 via various communication pathsor channels. Such paths or channels may include any type of datacommunication link known in the art, including TCP/IP connections andInternet connections via Wi-Fi, BLUETOOTH, a Universal MobileTelecommunications System (UMTS) network, or any other suitable datacommunication link. In that regard, network 175 may be a local areanetwork (LAN), which may be communicatively coupled to a wide areanetwork (WAN) such as the Internet. The Internet is a broad network ofinterconnected computers and servers allowing for the transmission andexchange of Internet Protocol (IP) data between users connected througha network service provider. Examples of network service providers arethe public switched telephone network, a cable service provider, aprovider of digital subscriber line (DSL) services, or a satelliteservice provider. Network 175 allows for communication between any ofthe various components of network environment 100.

Personal assistant server 180 is a server that manages functions of thepersonal assistants running on user devices 105 and 110 and is coupledto network 175 via network connection 170. Personal assistant server 180may be any suitable remote or local server that stores data. In someembodiments, personal assistant server 180 is an electronic storagedevice. Personal assistant server 180 is accessible by other devicesthrough network 175.

Personal assistant server 180 may include any type of server or othercomputing device as is known in the art, including standard hardwarecomputing components such as network and media interfaces,non-transitory computer-readable storage (memory), and processors forexecuting instructions or accessing information that may be stored inmemory. The functionalities of multiple servers may be integrated into asingle server. Alternatively, different functionalities may be allocatedamong multiple servers, which may be located remotely from each otherand communicate over the cloud. Any of the aforementioned servers (or anintegrated server) may take on certain client-side, cache, or proxyserver characteristics. These characteristics may depend on theparticular network placement of the server or certain configurations ofthe server.

FIG. 2 is a diagram illustrating exemplary settings 200 of an operatingsystem on a user device that may be used with a system for providingautomatic personal assistant features. In some embodiments, settings 200may be displayed on a user interface of respective user devices 105 and110 of FIG. 1. In some embodiments, settings 200 may correspond tosettings 125 and 130 of user devices 105 and 110 of FIG. 1. Settings 200may, for example, provide a mechanism by which a user may alter thefunctions of an operating system of a user device by implementingchanges to settings. Settings 200 may facilitate user interaction with auser device.

Settings 200 may include settings menu 205. Settings menu 205 mayinclude user-editable features for customizing the functionality of anoperating system or user device according to user preferences. In someimplementations, settings 125 and 130 of user devices 105 and 110 ofFIG. 1 to alter the performance of respective operating systems 115 and120. In some implementations, settings 125 and 130 of user devices 105and 110 of FIG. 1 to alter the performance of respective personalassistant applications. In some embodiments, settings 200 may bemodified by the user interacting with options or commands in arespective settings menu 205. Settings menu 205 may include any numberof user-selectable options or commands. Settings menu 205 may includeany suitable number of standard operating system or user devicesettings, for example, standard settings 210, including airplane mode,Wi-Fi, and cellular, as shown in FIG. 2. Standard settings 210 areexemplary interface elements that, when selected by a user, may, forexample, redirect the user to a respective new page, window, or dialoguebox.

In some embodiments, settings menu 205 includes a list ofuser-selectable options or settings presented in a hierarchical order.For example, automatic personal assistant settings 215 may besub-settings under standard settings 210. Standard settings 210 mayinclude automatic personal assistant settings 215, which is shown asselected (e.g., underlined) in FIG. 2, and the selection of automaticpersonal assistant settings 215 may reveal automatic personal assistancesettings 220-245. Automatic personal assistant settings 215 may includeexemplary settings categories that, when selected by a user, may, forexample, redirect the user to a respective new page, window, or dialoguebox. In another example, when selected, any of the interface elementsmay expand to reveal sub-options, sub-commands, or any other suitablesettings display elements.

In some embodiments, automatic personal assistant settings 215 mayinclude user-editable features for customizing the functionality of apersonal assistant application running on a user device. In someembodiments, automatic personal assistant settings 215 may be used tocustomize the functionality of personal assistant applications onrespective user devices 105 and 110 of FIG. 1. As illustrated in FIG. 2,automatic personal assistant settings 215 may include a mechanism forselection and de-selection of automatic personal assistance settings. Inthe shown embodiment, on/off selection buttons are illustrative examplesof mechanisms for selection and de-selection of automatic personalassistance settings. In some embodiments, selection and de-selection insettings menu 205 are binary selections.

In some embodiments, automatic personal assistant settings 215 includesa sub-menu of automatic personal assistance settings 220-245, which areuser-selectable options or commands for determining the functionality ofauto suggest software running on the user device. The automatic personalassistant settings 215 may include any suitable number of selectableautomatic personal assistance settings 220-245, which may correspond toexemplary data to be included or excluded from a status report, as shownin FIG. 2. In the illustrated embodiment, automatic personal assistantsettings 215 is selected to be “ON,” indicating the feature isactivated.

In the illustrated embodiment, exemplary automatic personal assistantsettings 235-245 are shown, including auto answer settings 220, autosearch settings 240, and auto load settings 245. Automatic personalassistant settings 220-245 may be used to allow or disallow interactionbetween a first personal assistant on a first user device and a secondpersonal assistant on a second user device. In some embodiments,automatic personal assistant settings 220-245 may be used to configureautomatic personal assistant features based on user preferences.

Auto answer settings 220 include any suitable number of selectablesub-settings for delivering an answer with a first personal assistant toa second personal assistant. Auto answer settings 220 may allow a userto configure a first personal assistant on a first user device toautomatically respond to incoming messages or commands from a seconduser device and/or a second personal assistant on the second userdevice. As shown, auto answer settings 220 include input type settings235, which may be used to define what types of automatic replies may beused by a personal assistant.

Auto answer settings 220 may also include with approval 225 and w/o(without) approval 230 options, which may allow a user to permitautomatic answers to different types of incoming communications with orwithout user approval of individual answers. In some embodiments, a usermay select with approval 225 option to permit a personal assistant toautomatically receive and process an input command from a third partydevice as well as identify a responsive action and/or answer but thepersonal assistant must prompt the user for permission for approvalbefore performing the responsive action and/or sending an answer. Insome embodiments, a user may select w/o approval 230 option to permitthe personal assistant to automatically perform the identified actionand/or send an answer without prompting the user for approval. Exemplaryinput type settings 235 are shown, including email, text, voicemail, andpersonal assistant. In some embodiments, an input type setting 235 isturned on, which allows a personal assistant to automatically answerincoming communications (i.e., input) of that type. For example, in theillustrated embodiment, email, text, and voicemail inputs are turned ONat with approval 225 and OFF at w/o approval 230, which may permit apersonal assistant to answer email, text, and voicemail input only withuser approval. User approval may be obtained, for example, via a promptdisplayed to the user when an input of any of these types is received orwhen provided by the user un-prompted. Also as shown, personal assistantinput is turned OFF at with approval 225 and ON at w/o approval 230,which may permit the personal assistant to automatically answer personalassistant input (i.e., from a second personal assistant on a second userdevice) without approval. It will be understood that the input typesshown are merely exemplary and that any suitable input types may beautomatically responded to by a personal assistant and associatedsettings configurable at input type settings 235.

Auto search settings 240 may allow the user to configure a personalassistant to automatically search information sources/databases on theuser device, including, for example, contacts, calendar, andgeolocation. In the illustrated embodiment, contacts, calendar, andgeolocation are all shown as ON. For example, a personal assistant mayreceive input from a second personal assistant, where the input is arequest for a phone number of a particular person. The personalassistant may, in response, automatically search the contacts storedlocally on the user device and automatically provide the contactinformation to the second personal assistant. A personal assistant mayautomatically search an allowed local information source based onkeywords parsed from received input.

Auto load settings 228 may allow a user to configure a personalassistant to automatically load information to the user device based onreceived input. Auto load settings 228 may allow automatic loading of,for example, a reminder entry or a calendar entry, both of which areshown as ON in FIG. 2. A personal assistant may automatically load anentry to the user device based on keywords parsed from received input.Automatically loaded entries may include any suitable data uploaded tothe user device and responsive to a received input.

It will be understood that the illustrated automatic personal assistancesettings are merely exemplary and not provided by way of limitation. Anysuitable settings for configuring automatic answers by a personalassistant may be used. For example, settings may also be used to setwhich types of outputs a personal assistant may use in automaticallyresponding to received input (e.g., email, text, phone call, or personalassistant command).

FIG. 3 is an illustrative table 300 of automatic answers 320 provided bya personal assistant in response to received inputs 315. Rows 330, 335,340, 345, and 350 respectively correspond to Events 305, E1-E5, eachshowing a different exemplary received text/email/voice/personalassistant input 315 sent from user Bob 310, and the correspondingtext/email/voice/personal assistant output 320 and local search or localload 325, as performed by the system. In some embodiments, receivedinputs 315 correspond to personal assistant input received at acommunications interface of a user device. For example, input generatedby a personal assistant on user device 110 may be received by a personalassistant on user device 105 via a user interface, as described above inconnection with FIG. 1.

Events E1 330, E2 335, and E3 340 present scenarios in which a personalassistant provides an answer to a received input based on a search,because the system identifies a keyword in the received input indicatingthat a search is necessary for responding to the received input. In someembodiments, the personal assistant may search any locally storeddatabase or information source on the user device as long as the usersettings do not disallow automatic searching of the database orinformation source. As described above in connection with FIG. 2,automatic personal assistant settings 215 may include auto searchsettings 240, by which a user may allow or disallow automatic searchesin local information sources, including email, contacts, and geolocationdata. For example, in Event E2 335, Bob may send a text message toCharlie [Charlie, do you know Paul Doe's phone #?], and Charlie'spersonal assistant may search Charlie's contacts 325 stored on Charlie'suser device and send a responsive text message [Yes, 555-555-5555] 320.In some embodiments, an automatic answer may include a local search, aresponsive communication, or both.

Events E4 345 and E5 350 present scenarios in which a personal assistantprovides an answer to a received input based on a data load, because thesystem identifies a keyword in the received input indicating that a dataload is necessary for responding to the received input. As describedabove in connection with FIG. 2, automatic personal assistant settings215 may include auto load settings 245, by which a user may allow ordisallow automatic loads, including, for example, reminder and calendarloads. For example, in Event E4 345, Bob may leave a voicemail messagefor Charlie [Charlie, remember to call mom tonight], and Charlie'spersonal assistant may perform a local load to add a reminder toCharlie's reminder application 325 [Tonight: Call mom reminder fromBob]. In some embodiments, an automatic answer may include a local load,a responsive communication, or both. FIG. 4, described below, is anillustrative table, which further defines example Events, as describedin connection with FIG. 3.

FIG. 4 is an illustrative table 400 of received inputs 404, localsearches 412, local loads 416, and automatic outputs 408. Illustrativetable 400 of FIG. 4 further defines exemplary Events described above inconnection with FIG. 3. Events E1-E5, shown in FIG. 4, correspond toevents 330-350 of FIG. 3, as described above. Table 400 providesexemplary combinations of types of received inputs 404, including email,text, voice, and personal assistant, and types of automatic outputs 408,including email, text, voice, personal assistant, and none. Localsearches 412 include calendar, contacts, geolocation, and other, andlocal loads 416 include reminder, calendar, and other.

Event E1 presents a scenario in which personal assistant command 420 isreceived input 404, including an indication that a local search of acalendar is needed, and personal assistant command 424 is provided asautomatic output 408. Event E2 presents a scenario in which text message428 is received input 404, including an indication that a local searchof contacts is needed, and text message 432 is provided as automaticoutput 408. Event E3 presents a scenario in which voicemail 436 isreceived input 404, including an indication that a local search ofgeolocation data is needed, and email 440 is provided as automaticoutput 408.

Event E4 presents a scenario in which voicemail 450 is received input404, including an indication that a local load of a reminder is needed,and no response 424 is provided as automatic output 408. Event E5presents a scenario in which email 448 is received input 404, includingan indication that a local load of a calendar entry is needed, andpersonal assistant command 452 is provided as automatic output 408.

It will be understood that the illustrated Events E1-E5 are merelyexemplary and not provided by way of limitation. It will be understoodthat the system may use any suitable combination of local searches 412and/or local loads 416 with received input 404 and/or automatic output408 to obtain a wide variety of possible events.

FIG. 5 is a flowchart illustrating an exemplary method 500 for providingautomatic personal assistance on first user device 512 in response to aninput communication received from second user device 508. In someembodiments, first user device 512 and second user device 508 maycorrespond to respective user devices 105 and 110 of FIG. 1. In someembodiments, first user device 512 includes first personal assistant405, and second user device 508 includes a second personal assistant(not shown). In some embodiments, first user device 512 and second userdevice 508 are smartphones.

In step 524, the operating system of first user device 512 activates anautomatic personal assistant feature of personal assistant 504. At step520, the system determines whether the automatic personal assistantfeature is activated. If the system determines the automatic personalassistant feature is activated, then it proceeds to step 532. If thesystem determines that the automatic personal assistant feature isdeactivated, then the system proceeds to end 528.

In step 532, the system parses the input. In some embodiments, step 532may be a first stage of the first personal assistant 504 parsing theinput. In some embodiments, step 532 may be performed by an inputsection of a set of routines in a software module. In some embodiments,in step 532 the system may identify keywords in a received inputcommunication and interact with a personal assistant server (e.g.,personal assistant server 180) over a network (e.g., network 175) inorder to determine a responsive action to the received input. If thesystem determines that a local load is necessary based on parsing thereceived input, the system will proceed to step 548. If the systemdetermines that a local search is necessary based on parsing thereceived input, the system will proceed to step 536.

In step 548, the system determines if a local load should be performed.If the system determines that the user has deactivated the local loadfeature in settings, the system will proceed to end 528. If the systemdetermines that the user has activated the local load feature insettings, the system may then determine if any entries need to be madeto local applications based on keywords identified in step 532. Forexample, the system may identify a calendar entry that needs to beloaded into the calendar application. If the system determines that alocal load needs to be performed, the system will perform the local loadin step 548. For example, the system will create a calendar entry andload it into the calendar application.

In step 536, the system determines if a local search should beperformed. If the system determines that the user has deactivated thelocal search feature in settings, the system will proceed to end 528. Ifthe system determines that the user has activated the local searchfeature in settings, the system may then determine if any searches needto be performed based on keywords identified in step 532. For example,the system may determine that a search of a calendar is needed todetermine the first user's availability. If the system determines that alocal search needs to be performed, the system will perform the localsearch in step 536.

In step 552, the system parses output by interacting with other localapplications (e.g., applications 140 of user device 105 of FIG. 1) inorder to determine a responsive action based on a local load on firstuser device 512. In some embodiments, step 552 may be implemented as anoutput section of a set of routines in a software module. For example,the system may extract meeting time data from a received inputcommunication and communicate with a calendar application so that ameeting entry may be loaded into the local calendar application. Thesystem may also insert an indication that the calendar entry was loadedto the calendar application into a responsive communication. In step556, a responsive output communication (i.e., Reply) is generated andsent over a network (e.g., network 175 of FIG. 1) to second user device508. In some embodiments, the responsive output may be a naturallanguage response. In some embodiments, the responsive output may beinclude information associated with a local load. For example, theresponsive output may include confirmation that a local load wasperformed.

In step 540, the system parses output by interacting with a personalassistant server (e.g., personal assistant server 180 of FIG. 1) inorder to determine a responsive action based on information gathered ina local search of first user device 512. In some embodiments, step 540may be implemented as an output section of a set of routines in asoftware module. For example, the system may extract a person's phonenumber from an entry in contacts stored in memory on first user device512 based on the person's name being mentioned and identified as akeyword in the received input communication. The system may also insertthat number into a reply text message. In step 544, a responsive outputcommunication (i.e., Reply) is generated and sent over a network (e.g.,network 175 of FIG. 1) to second user device 508. In some embodiments,the responsive output may be a natural language response. In someembodiments, the responsive output may be include information associatedwith a local search performed in step 536. For example, the responsiveoutput may include information extracted as a result of the localsearch.

Steps 556 and 544 may be implemented using any suitable communicationmedium, including email, voice, text message, or personal assistantcommand. In some embodiments, responsive output may be sent over anetwork, for example, network 175 of FIG. 1. In some embodiments, thesystem does not perform steps 556 or 544, and no responsive output isgenerated or sent to second user device 508.

FIG. 6 is a flowchart illustrating an exemplary method for providingautomatic personal assistance on a user device.

In step 610, the system receives user input, the user input includingautomatic personal assistant settings, at a user interface of a firstuser device. In some embodiments, the automatic personal assistantsettings may correspond to automatic personal assistant settings 215 asdescribed above in connection with FIG. 2.

In step 620, the system receives an input communication, at acommunication interface of the first user device, where the inputcommunication is sent from a second user device. In some embodiments,the input communication may be generated by a second personal assistanton the second user device. In some embodiments, the received input maybe communicated over a network, for example, network 175 of FIG. 1.

In step 630, the system parses the received input, with a first personalassistant on the first user device, based on the automatic personalassistant settings. In some embodiments, the first personal assistantmay parse the received input only if an automatic personal assistantfeature is turned ON in user settings of the first user device. In someembodiments, the first personal assistant processes the received inputto identify a responsive action to the received input.

In step 640, the system performs a local search or a local load based onthe parsing of the received input and on automatic personal assistantsettings. In some embodiments, the first personal assistant may performa responsive action, including a local search or a local load based onthe responsive action identified in step 630. In some embodiments, thescope of the local search (i.e., local data sources that can besearched) is based on user settings received in step 610. In someembodiments, the scope of the local load (i.e., what types of localloads may be loaded) is based on user settings received in step 610. Insome embodiments, the first personal assistant may perform both a localsearch and a local load when it determines, based on the parsing of thereceived input in 630, that both are necessary.

In step 650, the system sends a responsive output communication to thesecond user device. As described above in connection with FIG. 4, aresponsive output may include any suitable type of communication,including, for example, email, voice (e.g., phone call or voicemail),text message, and personal assistant command. In some embodiments, thefirst personal assistant does not send a responsive output communicationto the second user device. In some embodiments, the responsive outputmay be communicated over a network, for example, network 175 of FIG. 1.

FIG. 7 illustrates a mobile device architecture that may be utilized toimplement the various features and processes described herein.Architecture 700 can be implemented in any number of portable devicesincluding but not limited to smart phones, electronic tablets, andgaming devices. Architecture 700 as illustrated in FIG. 7 includesmemory interface 702, processors 704, and peripheral interface 706.Memory interface 702, processors 704 and peripherals interface 706 canbe separate components or can be integrated as a part of one or moreintegrated circuits. The various components can be coupled by one ormore communication buses or signal lines.

Processors 704 as illustrated in FIG. 7 is meant to be inclusive of dataprocessors, image processors, central processing unit, or any variety ofmulti-core processing devices. Any variety of sensors, external devices,and external subsystems can be coupled to peripherals interface 706 tofacilitate any number of functionalities within the architecture 700 ofthe exemplar mobile device. For example, motion sensor 710, light sensor712, and proximity sensor 714 can be coupled to peripherals interface706 to facilitate orientation, lighting, and proximity functions of themobile device. For example, light sensor 712 could be utilized tofacilitate adjusting the brightness of touch surface 746. Motion sensor710, which could be exemplified in the context of an accelerometer orgyroscope, could be utilized to detect movement and orientation of themobile device. Display objects or media could then be presentedaccording to a detected orientation (e.g., portrait or landscape).

Other sensors could be coupled to peripherals interface 706, such as atemperature sensor, a biometric sensor, or other sensing device tofacilitate corresponding functionalities. Location processor 715 (e.g.,a global positioning transceiver) can be coupled to peripheralsinterface 706 to allow for generation of geolocation data therebyfacilitating geo-positioning. An electronic magnetometer 716 such as anintegrated circuit chip could in turn be connected to peripheralsinterface 706 to provide data related to the direction of true magneticNorth whereby the mobile device could enjoy compass or directionalfunctionality. Camera subsystem 720 and an optical sensor 722 such as acharged coupled device (CCD) or a complementary metal-oxidesemiconductor (CMOS) optical sensor can facilitate camera functions suchas recording photographs and video clips.

Communication functionality can be facilitated through one or morecommunication subsystems 724, which may include one or more wirelesscommunication subsystems. Wireless communication subsystems 724 caninclude 802.x or Bluetooth transceivers as well as optical transceiverssuch as infrared. Wired communication system can include a port devicesuch as a Universal Serial Bus (USB) port or some other wired portconnection that can be used to establish a wired coupling to othercomputing devices such as network access devices, personal computers,printers, displays, or other processing devices capable of receiving ortransmitting data. The specific design and implementation ofcommunication subsystem 724 may depend on the communication network ormedium over which the device is intended to operate. For example, adevice may include wireless communication subsystem designed to operateover a global system for mobile communications (GSM) network, a GPRSnetwork, an enhanced data GSM environment (EDGE) network, 802.xcommunication networks, code division multiple access (CDMA) networks,or Bluetooth networks. Communication subsystem 724 may include hostingprotocols such that the device may be configured as a base station forother wireless devices. Communication subsystems can also allow thedevice to synchronize with a host device using one or more protocolssuch as TCP/IP, HTTP, or UDP.

Audio subsystem 726 can be coupled to a speaker 728 and one or moremicrophones 730 to facilitate voice-enabled functions. These functionsmight include voice recognition, voice replication, or digitalrecording. Audio subsystem 726 in conjunction may also encompasstraditional telephony functions.

I/O subsystem 740 may include touch controller 742 and/or other inputcontroller(s) 744. Touch controller 742 can be coupled to a touchsurface 746. Touch surface 746 and touch controller 742 may detectcontact and movement or break thereof using any of a number of touchsensitivity technologies, including but not limited to capacitive,resistive, infrared, or surface acoustic wave technologies. Otherproximity sensor arrays or elements for determining one or more pointsof contact with touch surface 746 may likewise be utilized. In oneimplementation, touch surface 746 can display virtual or soft buttonsand a virtual keyboard, which can be used as an input/output device bythe user.

Other input controllers 744 can be coupled to other input/controldevices 748 such as one or more buttons, rocker switches, thumb-wheels,infrared ports, USB ports, and/or a pointer device such as a stylus. Theone or more buttons (not shown) can include an up/down button for volumecontrol of speaker 728 and/or microphone 730. In some implementations,device 700 can include the functionality of an audio and/or videoplayback or recording device and may include a pin connector fortethering to other devices.

Memory interface 702 can be coupled to memory 750. Memory 750 caninclude high-speed random access memory or non-volatile memory such asmagnetic disk storage devices, optical storage devices, or flash memory.Memory 750 can store operating system 752, such as Darwin, RTXC, LINUX,UNIX, OS X, ANDROID, WINDOWS, or an embedded operating system such asVxWorks. Operating system 752 may include instructions for handlingbasic system services and for performing hardware dependent tasks. Insome implementations, operating system 752 can include a kernel.

Memory 750 may also store communication instructions 754 to facilitatecommunicating with other mobile computing devices or servers.Communication instructions 754 can also be used to select an operationalmode or communication medium for use by the device based on a geographiclocation, which could be obtained by the GPS/Navigation instructions768. Memory 750 may include graphical user interface instructions 756 tofacilitate graphic user interface processing such as the generation ofan interface; sensor processing instructions 758 to facilitatesensor-related processing and functions; phone instructions 760 tofacilitate phone-related processes and functions; electronic messaginginstructions 762 to facilitate electronic-messaging related processesand functions; web browsing instructions 764 to facilitate webbrowsing-related processes and functions; media processing instructions766 to facilitate media processing-related processes and functions;GPS/Navigation instructions 768 to facilitate GPS and navigation-relatedprocesses, camera instructions 770 to facilitate camera-relatedprocesses and functions; and instructions 772 for any other applicationthat may be operating on or in conjunction with the mobile computingdevice. Memory 750 may also store other software instructions forfacilitating other processes, features and applications, such asapplications related to navigation, social networking, location-basedservices or map displays.

Each of the above identified instructions and applications cancorrespond to a set of instructions for performing one or more functionsdescribed above. These instructions need not be implemented as separatesoftware programs, procedures, or modules. Memory 750 can includeadditional or fewer instructions. Furthermore, various functions of themobile device may be implemented in hardware and/or in software,including in one or more signal processing and/or application specificintegrated circuits.

Certain features may be implemented in a computer system that includes aback-end component, such as a data server, that includes a middlewarecomponent, such as an application server or an Internet server, or thatincludes a front-end component, such as a client computer having agraphical user interface or an Internet browser, or any combination ofthe foregoing. The components of the system can be connected by any formor medium of digital data communication such as a communication network.Some examples of communication networks include LAN, WAN and thecomputers and networks forming the Internet. The computer system caninclude clients and servers. A client and server are generally remotefrom each other and typically interact through a network. Therelationship of client and server arises by virtue of computer programsrunning on the respective computers and having a client-serverrelationship to each other.

One or more features or steps of the disclosed embodiments may beimplemented using an API that can define on or more parameters that arepassed between a calling application and other software code such as anoperating system, library routine, function that provides a service,that provides data, or that performs an operation or a computation. TheAPI can be implemented as one or more calls in program code that send orreceive one or more parameters through a parameter list or otherstructure based on a call convention defined in an API specificationdocument. A parameter can be a constant, a key, a data structure, anobject, an object class, a variable, a data type, a pointer, an array, alist, or another call. API calls and parameters can be implemented inany programming language. The programming language can define thevocabulary and calling convention that a programmer will employ toaccess functions supporting the API. In some implementations, an APIcall can report to an application the capabilities of a device runningthe application, such as input capability, output capability, processingcapability, power capability, and communications capability.

The foregoing detailed description of the technology herein has beenpresented for purposes of illustration and description. It is notintended to be exhaustive or to limit the technology to the precise formdisclosed. Many modifications and variations are possible in light ofthe above teachings. The described embodiments were chosen in order tobest explain the principles of the technology and its practicalapplication to thereby enable others skilled in the art to best utilizethe technology in various embodiments and with various modifications asare suited to the particular use contemplated. It is intended that thescope of the technology be defined by the claims appended hereto.

What is claimed is:
 1. A method for providing automatic personalassistance on a first user device, the method comprising: receiving avoice-based input communication at the first user device, the inputcommunication sent over a communication network from a second device;executing instructions stored in memory, wherein the execution ofinstructions by a processor executes the first personal assistant on thefirst user device to: activates an automatic mode of the first personalassistant, automatically parses the received voice-based inputcommunication to identify one or more keywords in the receivedvoice-based input communication, determines that the one or more parsedkeywords are associated with a responsive action, performs theidentified responsive action, and generates a voice-based outputcommunication responsive to the voice-based input communication, whereinthe voice-based output communication includes information associatedwith the performed responsive action; and transmits the voice-basedoutput communication over the communication network to the seconddevice.
 2. The method of claim 1, further comprising receiving thevoice-based user input at a user interface of the first user device,wherein the user input is associated with automatic personal assistantsettings, and wherein the automatic mode of the first personal assistantis activated based on the automatic personal assistant settings.
 3. Themethod of claim 2, wherein the automatic personal assistant settingsspecify types of received input communications for which the automaticmode of the first personal assistant can be activated.
 4. The method ofclaim 1, wherein the responsive action comprises searching data storedlocally in memory of the first user device.
 5. The method of claim 4,wherein the information associated with the performed responsive actionincludes search results.
 6. The method of claim 4, wherein the searcheddata comprises contacts data.
 7. The method of claim 4, wherein thesearched data comprises calendar data.
 8. The method of claim 4, whereinthe searched data comprises geolocation data.
 9. The method of claim 4,further comprising receiving the voice-based user input at a userinterface of the first device, wherein the voice-based user input isassociated with automatic search settings, and wherein the automaticsearch settings specify which of a plurality of local data sources areaccessible for search by the first personal assistant.
 10. The method ofclaim 1, wherein the responsive action comprises creating an entry in anapplication on the first user device by loading data from the receivedinput into the entry.
 11. The method of claim 10, wherein theinformation associated with the performed responsive action includesconfirmation that the entry has been created.
 12. The method of claim10, wherein the created entry comprises a calendar application entry.13. The method of claim 10, wherein, the created entry comprises areminder application entry.
 14. The method of claim 10, furthercomprising receiving the voice-based user input at a user interface ofthe first device, wherein the voice-based user input includes isassociated with automatic load settings, and wherein the automatic loadsettings specify for which of a plurality of applications the firstpersonal assistant is allowed to create entries.
 15. The method of claim1, wherein the first user device and the second user device are mobiledevices.
 16. A system for providing automatic personal assistance on afirst user device, the system comprising: a communication interface thatreceives a voice-based input communication at the first user device, thevoice-based input communication sent over a communication network from asecond device; a processor that executes the first personal assistant onthe first user device, wherein the execution of first personal assistantby the processor: activates an automatic mode of the first personalassistant, automatically parses the received voice-based inputcommunication to identify one or more keywords in the receivedvoice-based input communication, determines that the one or more parsedkeywords are associated with a responsive action, performs theidentified responsive action, and generates a voice-based outputcommunication responsive to the voice-based input communication, whereinthe voice-based output communication includes information associatedwith the performed responsive action; and wherein the communicationinterface transmits the voice-based output communication over thecommunication network to the second device.
 17. The system of claim 16,further comprising memory that stores data locally, wherein theresponsive action comprises searching the locally stored data.
 18. Thesystem of claim 16, wherein the responsive action comprises creating anentry in an application on the first user device by loading data fromthe received input into the entry.
 19. The system of claim 16, furthercomprising a user interface that receives user input regarding settings,and wherein the settings specify how the responsive action is identifiedor performed.
 20. A non-transitory computer-readable storage medium,having embodied thereon a program executable by a processor to provideautomatic personal assistance on a first user device, the methodcomprising: receiving a voice-based input communication, at acommunications interface on the first user device, the inputcommunication transmitted over a network from a second device;activating an automatic mode of a first personal assistant on the firstuser device; parsing the received voice-based input communication, bythe first personal assistant, to identify one or more keywords in thereceived input communication, the one or more keywords associated with aresponsive action, wherein the first personal assistant processes thereceived voice-based input communication automatically; performing theidentified responsive action, by the first personal assistant; andgenerating a voice-based output communication responsive to thevoice-based input communication, by the first personal assistant,wherein the output communication is transmitted over the network to thesecond device, and wherein the voice-based output communication includesinformation associated with the performed responsive action.